Computer-based automatic detection of an object touching a reference surface and/or determination of positional information (e.g., a coordinate in space) or motional information (e.g., velocity, acceleration) of the object finds a lot of applications in human-computer interactive entertainment and consumer electronics. In one such application, an interactive projection system for providing a display that interacts with a user is required to determine if a finger tip of the user touches a pre-defined region of a screen, in order that the interactive projection system receives user inputs. In another such application related to computer entertainment, the velocity of a user's finger striking a surface is used by a game to predict if the user is decisive or hesitant in providing an input to the game.
Recently, the emergence of wearable interactive-display devices, such as an interactive display integrated in an eyewear article, is shifting the paradigm of how a user interacts with the virtual and physical worlds. With a wearable interactive display for the user's eyes, the camera installed in the interactive display sees what the user sees. The user, hence, can link his or her interactions with the physical world to manipulate digital objects. However, such wearable device is usually worn on a part of the user's body to achieve mobility with the user. The relatively small display of the wearable device causes a limitation in the user-input means, and operation of the wearable device may cause inconvenience to the user. It is desirable and advantageous to obtain positional or motional information of the user's finger so as to allow the user of the wearable device to control digital information with his or her hands and fingers.
In US2014/0098224 (corresponding to U.S. patent application Ser. No. 14/102,506), it is disclosed a device that detects a height of a finger tip above a touch surface for determining if the finger touches the touch source so as to eventually determine a user input to the device. The device uses one image sensor, i.e. one camera, for detection of the finger tip's height above the touch surface. When the finger is illuminated, a shadow is cast on the touch surface. The camera is used to capture an image of the shadow. Based on the camera-observed shadow length, the finger tip's height above the touch surface is computed. However, when the detection method of this device is implemented in a wearable interactive display, there is a possibility that the finger tip's height above the touch surface is so large that it is difficult to detect the finger's shadow. It is desirable if the device is supplemented with an additional technique of detecting a 3D coordinate of the finger tip in air.
There is a need in the art for a wearable device that can detect a height of an object's tip above a surface if the surface is present, as well as can estimate a 3D coordinate of the tip in an absence of the surface.